New service: `DeviceRegistry` + new resources: `azurerm_device_registry_asset` and `azurerm_device_registry_asset_endpoint_profile` #28399

mryanlo · 2024-12-30T22:46:17Z

This PR introduces support for the Device Registry service, comprising of:

Resources

- azurerm_device_registry_asset
- azurerm_device_registry_asset_endpoint_profile

Community Note

Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for PR followers and do not help prioritize for review

Description

Swagger: https://github.com/Azure/azure-rest-api-specs/tree/main/specification/deviceregistry/resource-manager/Microsoft.DeviceRegistry/stable/2024-11-01

Azure Doc: https://learn.microsoft.com/en-us/azure/iot-operations/discover-manage-assets/howto-manage-assets-remotely?tabs=cli

PR Checklist

I have followed the guidelines in our Contributing Documentation.
I have checked to ensure there aren't other open Pull Requests for the same update/change.
I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
For example: “resource_name_here - description of change e.g. adding property new_property_name_here”

Changes to existing Resource / Data Source

I have added an explanation of what my changes do and why I'd like you to include them (This may be covered by linking to an issue above, but may benefit from additional explanation).
I have written new tests for my resource or datasource changes & updated any relevent documentation.
I have successfully run tests with my changes locally. If not, please provide details on testing challenges that prevented you running the tests.
(For changes that include a state migration only). I have manually tested the migration path between relevant versions of the provider.

Testing

My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

The acceptance tests for azurerm_device_registry_asset and azurerm_device_registry_asset_endpoint_profile require an arc-enabled kubernetes cluster with Azure IoT Operations' service extension installed on the cluster in order to pass. Creating this cluster in the terraform acceptance tests has been unsuccessful, takes over 45 minutes to complete, and the installation of AIO script has proven to be flakey. Once the cluster is setup, you need to add these environment variables to pass:

ARM_DEVICE_REGISTRY_RESOURCE_GROUP=<myResourceGroup> // resource group where the AIO-enabled cluster is
ARM_DEVICE_REGISTRY_CUSTOM_LOCATION=/subscriptions/xxxx/resourceGroups/<myResourceGroup>/providers/Microsoft.ExtendedLocation/customLocations/<myCustomLocation> // full custom location path that is in <myResourceGroup> and connected to the AIO extension

Local tests against a pre-existing AIO cluster:

Asset:

$ make acctests SERVICE='deviceregistry' TESTARGS='-run=TestAccAsset_'
TF_ACC=1 go test -v ./internal/services/deviceregistry -run=TestAccAsset_ -timeout 180m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccAsset_basic
=== PAUSE TestAccAsset_basic
=== RUN   TestAccAsset_complete
=== PAUSE TestAccAsset_complete
=== RUN   TestAccAsset_requiresImport
=== PAUSE TestAccAsset_requiresImport
=== RUN   TestAccAsset_update
=== PAUSE TestAccAsset_update
=== CONT  TestAccAsset_basic
=== CONT  TestAccAsset_requiresImport
--- PASS: TestAccAsset_requiresImport (148.12s)
=== CONT  TestAccAsset_update
--- PASS: TestAccAsset_basic (168.96s)
=== CONT  TestAccAsset_complete
--- PASS: TestAccAsset_complete (125.88s)
--- PASS: TestAccAsset_update (181.90s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/deviceregistry        340.311s

Asset endpoint profile:

$ make acctests SERVICE='deviceregistry' TESTARGS='-run=TestAccAssetEndpointProfile_'
TF_ACC=1 go test -v ./internal/services/deviceregistry -run=TestAccAssetEndpointProfile_ -timeout 180m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccAssetEndpointProfile_basic
=== PAUSE TestAccAssetEndpointProfile_basic
=== RUN   TestAccAssetEndpointProfile_complete_certificate
=== PAUSE TestAccAssetEndpointProfile_complete_certificate
=== RUN   TestAccAssetEndpointProfile_complete_usernamePassword
=== PAUSE TestAccAssetEndpointProfile_complete_usernamePassword
=== RUN   TestAccAssetEndpointProfile_complete_anonymous
=== PAUSE TestAccAssetEndpointProfile_complete_anonymous
=== RUN   TestAccAssetEndpointProfile_requiresImport
=== PAUSE TestAccAssetEndpointProfile_requiresImport
=== RUN   TestAccAssetEndpointProfile_update
=== PAUSE TestAccAssetEndpointProfile_update
=== CONT  TestAccAssetEndpointProfile_basic
=== CONT  TestAccAssetEndpointProfile_complete_anonymous
--- PASS: TestAccAssetEndpointProfile_basic (141.14s)
=== CONT  TestAccAssetEndpointProfile_update
--- PASS: TestAccAssetEndpointProfile_complete_anonymous (146.94s)
=== CONT  TestAccAssetEndpointProfile_requiresImport
--- PASS: TestAccAssetEndpointProfile_requiresImport (113.54s)
=== CONT  TestAccAssetEndpointProfile_complete_usernamePassword
--- PASS: TestAccAssetEndpointProfile_complete_usernamePassword (118.11s)
=== CONT  TestAccAssetEndpointProfile_complete_certificate
--- PASS: TestAccAssetEndpointProfile_complete_certificate (118.48s)
--- PASS: TestAccAssetEndpointProfile_update (379.77s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/deviceregistry        531.097s

Both resources' tests skipped if env variables not set:

 make acctests SERVICE='deviceregistry'
TF_ACC=1 go test -v ./internal/services/deviceregistry  -timeout 180m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccAsset_basic
    device_registry_asset_resource_test.go:30: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAsset_basic (0.00s)
=== RUN   TestAccAsset_complete
    device_registry_asset_resource_test.go:57: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAsset_complete (0.00s)
=== RUN   TestAccAsset_requiresImport
    device_registry_asset_resource_test.go:127: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAsset_requiresImport (0.00s)
=== RUN   TestAccAsset_update
    device_registry_asset_resource_test.go:148: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAsset_update (0.00s)
=== RUN   TestAccAssetEndpointProfile_basic
    device_registry_assetendpointprofile_resource_test.go:30: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_basic (0.00s)
=== RUN   TestAccAssetEndpointProfile_complete_certificate
    device_registry_assetendpointprofile_resource_test.go:57: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_complete_certificate (0.00s)
=== RUN   TestAccAssetEndpointProfile_complete_usernamePassword
    device_registry_assetendpointprofile_resource_test.go:85: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_complete_usernamePassword (0.00s)
=== RUN   TestAccAssetEndpointProfile_complete_anonymous
    device_registry_assetendpointprofile_resource_test.go:113: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_complete_anonymous (0.00s)
=== RUN   TestAccAssetEndpointProfile_requiresImport
    device_registry_assetendpointprofile_resource_test.go:141: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_requiresImport (0.00s)
=== RUN   TestAccAssetEndpointProfile_update
    device_registry_assetendpointprofile_resource_test.go:162: Skipping test due to missing environment variables ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and/or ARM_DEVICE_REGISTRY_RESOURCE_GROUP
--- SKIP: TestAccAssetEndpointProfile_update (0.00s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/deviceregistry        13.818s

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

new service DeviceRegistry - support for the device_registry service [GH-00000]
new resource azurerm_device_registry_asset [GH-00000]
new resource azurerm_device_registry_asset_endpoint_profile [GH-00000]

This is a (please select all that apply):

Bug Fix
New Feature (ie adding a service, resource, or data source)
Enhancement
Breaking Change

Related Issue(s)

Fixes #0000

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

…eregistry tf service

…d only fields per terraform requirement

…azurerm into rylo/adr-terraform

…to local cluster only

…azurerm into rylo/adr-terraform

…ient registration

mryanlo · 2025-02-21T22:55:51Z

internal/services/deviceregistry/device_registry_asset_resource_test.go

+	r := AssetTestResource{}
+
+	if os.Getenv(ASSET_CUSTOM_LOCATION_NAME) == "" || os.Getenv(ASSET_RESOURCE_GROUP_NAME) == "" {
+		t.Skipf("Skipping test due to missing environment variables %s and/or %s", ASSET_CUSTOM_LOCATION_NAME, ASSET_RESOURCE_GROUP_NAME)


For both asset and asset endpoint profile resource tests, we require env vars ARM_DEVICE_REGISTRY_CUSTOM_LOCATION and ARM_DEVICE_REGISTRY_RESOURCE_GROUP to be set or else the tests will fail (to prevent this we skip the tests).

however, the resource group, custom location, and the AIO cluster must be pre-existing this test run. Investigation and efforts to get the acceptance tests to create and provision an AIO cluster are flakey, take a long time to run (45+ minutes), and we found that the test steps delete the cluster resources before the asset/asset endpoint profile resource tests run (causing the latter ones to fail). awaiting response from Hashicorp and Terraform PMs what the best course of action is here.

Hey @mryanlo,

In short it would be acceptable for the tests to run against existing infrastructure for the time being to help move this PR forwards, but the maintainers do need to be able to independently run the acceptance tests for new RPs and resources that are in the provider. I'd like to better understand the procedure required to test these resources so that we can setup the tests to run without a dependence on pre-existing resources.

We have acceptance tests in the provider for arc enabled clusters where we need to provision the prerequisite network resources and a VM that runs the necessary setup scripts for the cluster.

Would you be able to take a look and let me know if there are parts of this setup that you would be able to re-use and what modifications you would need to make for setting up AIO on the cluster specifically? e.g. perhaps the setup script etc.

Once we have a better understanding of what's required we can begin looking at ways to cut down the provisioning time and improving the stability of the test.

Hi @stephybun thanks for reaching out! the exact steps to install AIO onto a cluster are found here: https://review.learn.microsoft.com/en-us/azure/iot-operations/get-started-end-to-end-sample/quickstart-deploy?branch=main

I've seen that link you sent me in the past and have been unable to successfully modify it for AIO's purposes. This is due to the complexity and flakiness of the bash/python script that the VM must run, and our restrictions with the linux distro for installing AIO (you need linux Ubuntu 24.04 with a k3s cluster, not kind cluster to succeed). Additionally there are a bunch of az cli commands that must run and this process is the only available way to install AIO on a cluster, so we need to install az cli and k3s.

If you'd like to see my latest attempt at modifying the scripts to get these to work (the tests will try to sequentially first create a single cluster with AIO installed and then run the acc tests to create resources on only that cluster), here are links:

https://github.com/mryanlo/terraform-provider-azurerm/blob/rylo/adr-tf-tests/internal/services/deviceregistry/device_registry_assetendpointprofile_resource_test.go
https://github.com/mryanlo/terraform-provider-azurerm/tree/rylo/adr-tf-tests/internal/services/deviceregistry/testdata
Thanks, and let me know if you have any more questions about this process.

Note: my code modifies the acceptance tests to run via a managed identity since due to security reasons we are not allowed to make service principals to run terraform anymore.

@stephybun is there a way to ignore the terraform plan check in acceptance tests from running on the VM portions? we don't care about the VM changing since it's not the resource that we want to test. however, one of the issues with my script above is that even if the AIO installation succeeds, the test flags the terraform plan step of the VM as modified after creating.

mryanlo · 2025-03-03T18:56:33Z

internal/services/deviceregistry/device_registry_asset_resource.go

+					param.Properties.DefaultTopic.Path = pointer.To(config.DefaultTopicPath)
+				}
+				if defaultTopicRetainChanged {
+					// Bug with `go-azure-sdk` library: you can't set retain to null because empty string will cause


Linking issue hashicorp/go-azure-sdk#1143 and PR hashicorp/pandora#4633

The go sdk does not support sending null properties. After the PR goes in, this issue can be fixed but might require a code change here

mryanlo added 3 commits December 25, 2024 15:31

boilerplate service code

aab6731

boilerplate service package

4c3f492

AEP resource

d68570d

github-actions bot added the size/XL label Dec 30, 2024

mryanlo added 3 commits December 30, 2024 15:20

fix build errors and lints

9608e73

lint

a8d1a18

cleanup and lint for AEP

9397281

mryanlo changed the title ~~New service: DeviceRegistry~~ DO NOT REVIEW!!! New service: DeviceRegistry Dec 30, 2024

mryanlo marked this pull request as ready for review December 30, 2024 23:55

mryanlo requested a review from a team as a code owner December 30, 2024 23:55

mryanlo added 5 commits December 30, 2024 19:15

update aep + asset resource

96b2367

vendor dependencies

4c3891e

update codeowners to add device registry required reviewers for devic…

8c9fdd8

…eregistry tf service

fix build errors in aep and asset resource types

7a1b1a5

remove untyped service

287a79c

github-actions bot added the dependencies label Dec 31, 2024

run go generate ./internal/provider

32aced7

github-actions bot added the documentation label Dec 31, 2024

mryanlo marked this pull request as draft January 24, 2025 01:32

mryanlo added 11 commits January 28, 2025 15:54

documentation for asset and AEP resources, removing asset and AEP rea…

2033c9a

…d only fields per terraform requirement

fix aep resource bugs

6d24814

simplify update logic for aep auth methods

3ef00f4

fix bugs in asset resource

54394e5

documentation example updates

bdb3547

remove attributes fields from documentation

5435de0

rough draft of tests

e3ef30f

Merge branch 'main' of https://github.com/mryanlo/terraform-provider-…

36600f5

…azurerm into rylo/adr-terraform

add kind.yaml

09a5eb4

snake case aep names to fix validation errors

547600d

lint

62e4291

mryanlo added 2 commits February 18, 2025 19:27

delete aio cluster setup scripts, make asset tests, change aep tests …

4ec9666

…to local cluster only

add comment

505965b

mryanlo changed the title ~~DO NOT REVIEW!!! New service: DeviceRegistry~~ New service: DeviceRegistry Feb 19, 2025

mryanlo changed the title ~~New service: DeviceRegistry~~ New service: DeviceRegistry + new resources: azurerm_device_registry_asset and azurerm_device_registry_asset_endpoint_profile Feb 19, 2025

add back documentation uri update asset

f2669aa

mryanlo marked this pull request as ready for review February 19, 2025 08:25

mryanlo added 3 commits February 19, 2025 00:47

fix asset test

a38f5c7

Merge branch 'main' of https://github.com/mryanlo/terraform-provider-…

e877e4f

…azurerm into rylo/adr-terraform

add github label issue triage

546a79f

github-actions bot added the waiting-response label Feb 20, 2025

mryanlo added 2 commits February 20, 2025 11:05

fix lint issues

eab55f2

fix github labeler triage bug by adding the github label method to cl…

31c12f7

…ient registration

github-actions bot removed the waiting-response label Feb 20, 2025

mryanlo commented Feb 21, 2025

View reviewed changes

mryanlo added 3 commits February 25, 2025 14:30

update codeowners

cd6c6f9

change resource logic to poll for create and update functions

4dbcf64

fix build

41e54a3

mryanlo commented Mar 3, 2025

View reviewed changes

remove comments

f012f22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New service: `DeviceRegistry` + new resources: `azurerm_device_registry_asset` and `azurerm_device_registry_asset_endpoint_profile` #28399

New service: `DeviceRegistry` + new resources: `azurerm_device_registry_asset` and `azurerm_device_registry_asset_endpoint_profile` #28399

mryanlo commented Dec 30, 2024 •

edited

Loading

mryanlo Feb 21, 2025

stephybun Mar 4, 2025

mryanlo Mar 4, 2025 •

edited

Loading

mryanlo Mar 4, 2025

mryanlo Mar 7, 2025

mryanlo Mar 3, 2025

New service: DeviceRegistry + new resources: azurerm_device_registry_asset and azurerm_device_registry_asset_endpoint_profile #28399

Are you sure you want to change the base?

New service: DeviceRegistry + new resources: azurerm_device_registry_asset and azurerm_device_registry_asset_endpoint_profile #28399

Conversation

mryanlo commented Dec 30, 2024 • edited Loading

Resources

Community Note

Description

PR Checklist

Changes to existing Resource / Data Source

Testing

Change Log

Related Issue(s)

mryanlo Feb 21, 2025

Choose a reason for hiding this comment

stephybun Mar 4, 2025

Choose a reason for hiding this comment

mryanlo Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

mryanlo Mar 4, 2025

Choose a reason for hiding this comment

mryanlo Mar 7, 2025

Choose a reason for hiding this comment

mryanlo Mar 3, 2025

Choose a reason for hiding this comment

New service: `DeviceRegistry` + new resources: `azurerm_device_registry_asset` and `azurerm_device_registry_asset_endpoint_profile` #28399

New service: `DeviceRegistry` + new resources: `azurerm_device_registry_asset` and `azurerm_device_registry_asset_endpoint_profile` #28399

mryanlo commented Dec 30, 2024 •

edited

Loading

mryanlo Mar 4, 2025 •

edited

Loading